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This working paper summarises a very 
interesting study conducted on over 
30,000 students in India’s leading 
schools in 2006. The results suggest 
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Educational Initiatives (El) believes that significantly improved student learning can happen only through systematic research into learning 
which includes assessment , as well as areas like misconception research. This working paper series will share learnings from various past and 
present El projects as well as path-breaking work in these areas elsewhere in the world. Please write to us at assessment(ci)ei-india.com for 

questions or comments. 
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■ WHY THIS PROJECT HAS LARGER RELEVANCE 


How well are students in our ‘top’ schools learning? In order to try and understand this, Educational Initiatives (El) and Wipro Ltd. conducted 
a research study in the 5 metros - Mumbai, Kolkata, Chennai, Delhi and Bangalore in 2006. 

These ‘top’ schools in our largest cities arguably represent the best education that is available in the country. These schools are likely to be 
the least constrained for different types of resources. Facilities like libraries, computers, playgrounds as well as access to resources (being 
located in the metros) are the best in the country. Partly for these reasons, these schools, in general, are the most coveted among parents 
and serve as role models to other schools. Often they are the thought leaders and their ideas and practices set the tone for others. It is 
therefore interesting to study how these ‘role models’ are faring. 

In a limited sense, the purpose of this study was to understand how well students are learning in our ‘top’ schools. However the larger goal 
was to take a critical issue - the quality of student learning - and to try and gain insights about it based, not on opinion or ‘experience’, but on 
hard data. 


■ SALIENT FEATURES OF THE STUDY 

• Expert Panel: A group of eminent educationists and school principals formed an expert 
panel that oversaw and was consulted on various aspects of the study. 

• Selection of Top English Medium Schools: The ‘top’ 40 schools were identified 
through a popular survey in each metro and then invited to participate in the study. In 
each metro, the survey covered about 200 people across 10 categories based on their 
profession and location in different geographical areas. 

• Specially Assembled Test Paper: Students of classes 4, 6 and 8 from these schools 
were assessed through a test. The test consisted of objective, multiple-choice questions 
in English, Mathematics, and Science as well as a writing task. The questions were 
carefully selected from a pool of ASSET items which have already been extensively 
tested with thousands of students. 

• International Benchmarking: Some questions from an international test, the TIMSS, were also included in this to provide a 
comparison with the performance of international students. 

• A Writing Task: All students were administered a writing test, where they had to write a paragraph, story or report. This revealed 
insights about the writing competencies of students in these top schools. 

• Completely Invigilated Tests: All the tests were invigilated by El representatives, to ensure that the processes were standardised 
and possibilities of copying or collusion were minimised. 

• Background Questionnaires: The test papers given to the students include 4 questions seeking information about student 
preferences and interests. School principals were also asked to fill a questionnaire seeking information about the school and its 
facilities. 

• Analysis: Different types of analyses were carried out on the collected data to extract patterns in performances and to understand 
differences in learning levels across different groups. Advanced statistical methods and Item Response Theory were used to confirm 
patterns. Analysis was also done to check if copying had occurred. 


Number of students schools who took the tests in the different cities 



Bangalore 

Chennai 

Delhi 

Kolkata 

Mumbai 

Total 

Schools Tested 

26 

37 

24 

23 

32 

142 

Students tested 
(Main Study) 

3286 

4738 

2814 

2555 

3845 

17238 

Students tested 
(Secondary Study)* 

2996 

4424 

2610 

2529 

3889 

16448 


*The secondary study used common papers for classes 4, 6 and 8. 



■ QUESTIONS THAT CHECK UNDERSTANDING 


In this test, students were assessed on concepts that are 
part of their curriculum. However, the test used questions 
that checked for conceptual understanding, unlike the 
standard school tests which generally check rote 
learning. For example, the universalities of laws and 
rules is also often not understood by students. As shown 
in the example alongside. 


"When light is reflected from a smooth surface, the angle of incidence must be equal to the angle of 
reflection.” 


i 


U n der which of th e f ollowin g situat ions is. the above statement tru e? 

A . 3 1 is a Iways true. t2 6%) 

B . ft is true only i f the surface is €0 N C A V E or PLANE. (17%) 

C. 3 1 is true only If the surface is made of GLASS (1314) 

D. It is true only if the surface is PLANE, (42%) 
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■ MAIN FINDINGS 


1. Students across classes answer rote-based or procedural questions relatively well. The flip side of this, however, is that students 
seem to rely on memory or learnt procedures to answer almost all questions, rather than trying to think through and solve the 
unfamiliar ones. 

2. It appears that many practical competencies, important in real life, are not being developed very well. Students’ performance on 
questions based on measurement, estimation, problem solving, general observation and day-to-day language use is not very good. 

3. Instead of acquiring concepts, students seem to be learning to handle a limited number of question types. So when they come across 
a question similar to one they have ‘learnt’, they ‘jump’ to the most familiar answer they find! 

4. Students are learning in ‘compartments', i.e. they may be aware of two pieces of information, but often not know how they are related 
or how that relation works in a real life situation. 

5. Apart from problems with learning strategies, a number of specific and clear ‘common errors’ exist in the different subjects. Since they 
are widespread, it should not be difficult for textbooks and teachers to specifically address these errors. 

6. Some background information - like the average class size, the number of hours of instruction per week, the number of working days 
per week and whether the school has laboratories for English, Maths and Science - was sought from each participating school 
through a questionnaire developed for the purpose. Only a very weak negative correlation was observed with the class strength. No 
other factors correlated significantly with the average performance of the schools/classes. 


■ SUBJECT-WISE FINDINGS - MATHS 


• Analysis shows that conceptual understanding tends to be weak in certain topics like decimal fractions, concepts like perimeter and 
general number sense. Students do not seem to have developed a conceptual understanding of decimal fractions. Similarly, a 
common misconception is that multiplication must always yield a greater number than division or other operations. Students seem 
shaky with the concepts of area and perimeter. Even among the two, perimeter seems to be less understood. 


MATHS 

WHAT IS THE LENGTH OF THE 
PENCIL IN THE PICTURE? 



Kr 

i‘T'Q\ 

A 4 cm 

G 

6 cm 

B 5 cm 

• 

7 cm 

CORRECT ANSWER IS OPTION *B’ 

How they answered 1 


A 7*- 
B 11* 
c 77* 
d 3* 

| Right answer | Wrong answers 


Observation 

Class IV students know that 
readings have to be taken 
from the right side of the scale 
but not when the left end is 
not at zero. That this is not 
just an oversight is evident 
because the error is repeated 
in similar questions. 



Graphcs by ALANKAH 
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IDENTIFY THE CORRECT ARRANGEMENT 

Three solids when arranged one on top of the other 
look like this from the top (see figure ). 



Which of these 
could be the 
arrangement? 



A ffii 



• • 



Observation 

Class VI fared 


relatively well in a 
question needing 
visualisation even 
though this is not 


| Right answer 
^ Wrong answers 


formally taught in 


CORRECT ANSWER IS OPTION ‘B’ 


schools. 


Percentages in this graphic and others may not total 100 because of invalid or no responses 


WHAT IS THE PERIMETER? 

Nandita cut off a square of side 1 cm from a rectan- 
gular sheet as follows. What would be the change in 
perimeter as compared to the original paper? 


* 



How they answered 



Observation 


| Right answer 
| Wrong answers 


Class VIII 
students seemed 


A 

B 

G 




It would remain the same 
It would increase by 2 cm 
It would decrease by 2 cm 
It would decrease by 3 cm 

CORRECT ANSWER IS OPTION ‘B’ 


to think that 
cutting something 
off will mean the 
perimeter will 
reduce. 
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■ SUBJECT-WISE FINDINGS - ENGLISH 


• A number of basic errors related to spellings and pronunciation were noticed in which 
students were asked to identify a word rhyming with ‘dawned’ in class 8. Similarly, 
students chose ‘hear’ (60%) rather than ‘pear’ (22%) when asked which word rhymes 
with ‘hair’. They also felt that ‘size’ rhymes with ‘seize’, not ‘flies'. It is clear that many 
students are choosing words that are spelt similarly, rather than rhyming words. 

• Comprehension errors are common in cases where the answer does not appear explicitly 
in the text. 


■ STUDENTS’ ENGLISH WRITING SKILLS 


In addition to the objective type items used in the main and secondary studies, a writing task - 
writing a paragraph, describing a picture/incident or writing a story - was also given to each 
student. About 32,000 student writing samples were thus obtained. A random sample of these 
were selected and analysed. 

The analysis shows that although students’ proficiency in English increases with class, even in 
class 8 about 80% of students are making mistakes in comprehension, grammar and syntax. It 
appears that for a large population of students language learning is mechanical, with students 
often failing to relate a writing task to its real life implications. 


WHAT’S AMISS IN ENGLISH 

Common errors that occur in student writing across classes IV, VI and VIII 




earring for carrying 
gowing for going 
sum for some 

say the aunty, said you, asked that, he said his mother 

is been carrying, has went 

Ignoring capitalisation of proper nouns 

Errors in sentence formation and punctuation 

minites for minutes 
carring for carrying 

said you , not in the home 

Improper use of capitals and apostrophe and 
erroneous sentence formation 

has went , is been carrying t was come , cafts to our 
house 

carring for carrying 
recieve for receive 
aunti/anti for aunty 
meat for meet 
greatfut for grateful 



asked to me, rang of the phone, the Dipti Aunty 

Ignoring capitalisation of proper nouns and 
unnecessary speech marks in written text 

if you going to be delayed 


Onus. UveA. aW>^ TWiA-ua. . 


ac& A Vu -£> mem’s 


iJoEffi 
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some v<»r Wrvi 'Dicta u^nn Vn 
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Some students wrote stories where they had to report a telephonic message. 




ENGLISH 

All questions© EVEI a I 

CHOOSE THE SENTENCE WITH 
NO MISTAKES. 

A The taj mahal is is agra 
[ 5 The Taj Mahal is in agra 
(1 The Taj mahal is in agra 

I ) The Taj Mahal is in Agra 

CORRECT ANSWER IS OPTION ‘D f 


How they answered 


A 6 s 
B 6* 

c 6* 

o 80 s 

■ Right answer 


Observation 

Grammar tends to be empha- 
sized a lot in our system — one 
reason why 80 per cent stu- 
dents of class IV got this right. 


WHY, ACCORDING TO THE 
PASSAGE, IS PHARAOH 
TUTANKHAMEN FAMOUS? 

"Tutankhamen’s 
tomb was forgotten 
until Howard Carter 
discovered it... (in) 

1922,., (The) trea- 
sures (in his tomb) 
made Tutankhamen 
perhaps the best 
known of the pharaohs..." 

A He died under tragic conditions 
| ] His tomb had many valuables 
( ; The walls of his tomb were thick 
| } The tomb was discovered in 1922 
CORRECT ANSWER IS OPTION ‘B’ 


How they answered 


A 8* - 
B 

c 8 s 
d 37* 

| Right answer 
| Wrong answers 


Observation 

None of the phrases in the 
question appear as they do 
in the passage — enough to 
confuse class VI students. 





| Wrong answers 


o 
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■ SUBJECT-WISE FINDINGS - SCIENCE 


SCIENCE 


WHAT IS THE CHEMICAL 
FORMULA OF PURE STEAM? 



I ) Pure steam does not have a 
chemical formula 

CORRECT ANSWER IS OPTION J B f 



Observation 


Class VIII students fail to 
make a two-step reasoning: 
that water is H2O and it 
evaporates to form steam. 


All q uestio ns © f Jfellj l 



THE EARTH S ROTATION ON ITS 
AXIS IS THE REASON FOR... 


1 5 longer days in summer 
(! eclipses 
I ) day and night 


CORRECT ANSWER IS OPTION ‘D' 


How they answered 


A 17 * 

B 8* 

c7* - 

d 66* - 

■ Right answer 


Observation 

Class IV students fare well in 
this typically ‘textbookish’ 
question showing ease in 
handling rote problems — this 
is seen in other such 
questions as well across all 
grades and in all cities. 



In Science, misconceptions or common 
errors were found. Some of these are:- 

A. When asked the chemical formula for pure 
steam, 51% students answered that pure 
steam does not have a chemical formula, 
even though the 3 facts given below are 
known to the students by this stage: 

1 . The chemical formula of water is H 2 0. 

2. Water changing to steam is a physical 
change 

3. Chemical composition does not change 
in a physical change. 

B. If one considers biologically, is Man an 
animal? A fly? The answer is yes, and it is 
discussed under classification of living things 
- an important topic. Yet 19% of class 8 
students do not think that Man is an animal 
and 47% do not think flies are animals. 

C. The height of the water collected (though 
not the volume) in any vessel of uniform 
cross-section in a rain will be the same - that 
is why rainfall is measured in cm. Yet, 74% 
of students in class 6 did not think so. This 
kind of question will be answered well if 
students are required to do simple 
experiments like collecting rain water and 
checking what the height of water collected 
depends on. 


■ COMPARATIVE PERFORMANCE OF THE METROS 

It may be expected that the ‘top’ schools of the 5 metros will NOT show any major difference in their overall performance. After all, many 
important factors are similar across the schools. These include factors like infrastructure, socio-economic background of the students, and 
teacher profile. But the data showed clear differences between the performances of schools in these metro cities. Top schools of Delhi, 
Mumbai and Kolkata performed significantly better than those of both Chennai and Bangalore. This result was cross checked in three 
different ways for robustness of the pattern found - performance in the Secondary Study, by removing outliers in the main study and 
checking if there is an effect due to some top schools refusing to participate in some cities. All 3 analyses showed the same trend, with 
Chennai and Bangalore performing lower than other cities. Please note that all this refers only to the performance of “top schools” - for 
example, no conclusion can be drawn about the comparative performance of government schools in these cities. 


Comparative Performance of the Different Cities (Main Study) 


.<2 

■o 


c 

ro 

<75 



Delhi 


Kolkata 


Mumbai 


■ English 

■ Maths 

ii Science 
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■ COMPARISON WITH INTERNATIONAL BENCHMARKS 


How are the students in the ‘top’ schools of our metros performing compared to their peers in other countries? To find this out, some 
questions from Trends in International Mathematics and Science Study (TIMSS), an international assessment in which over 40 
countries participated were included in the paper used for the secondary study. Since the performance of students internationally on these 
questions was known, a comparative picture could be obtained. 

The findings are, to say the least, quite worrying. Students from the ‘top’ schools of our metros are performing below average international 
levels. The performance of the class 4 students is significantly lower than the international average in each one of the 1 1 questions included 
from TIMSS. More strikingly, students of a higher class (class 6) performed only marginally better than the international average for class 4 
students. As seen in the graph, the trend is similar in both Maths and Science. It is worth remembering that the data being compared is of 
Indian students in ‘top’ schools (by popular perception) and the average students of other countries. 


r 



Performance of Indian Students compared 
to students from 43 countries 
(SCIENCE) 



J 




Performance of Indian Students compared 
to students from 43 countries 
(MATHS) 

IOO 

80 
60 
40 
20 
0 

(TIMSS) (TIMSS) (Metro) (Metro) (Metro) 



J 



040 Knowing how insects grow and transform 



73% of metro students in class 4 got this right , which is good, but internationally, 82% of class 3 and 85% of class 4 students answered correctly 


■ OVERALL RECOMMENDATIONS 

This section contains some specific recommendations mostly at a systemic level. Some are easier to implement than others, but all of them, 
we believe, would help us move closer to the goal of improved student learning. 

1 . Stereotypical misconceptions / mistakes in comprehension exist among students - many of which are highlighted in this report. When 
updating textbooks and during teacher training programmes, such findings should be taken into account so that the incidence of such 
misconceptions may reduce. 

2. A debate needs to be initiated on the importance and role of learning with understanding and the problems with rote learning. 

3. The nature of questions in the Board Exams should change with more questions that test learning with understanding and fewer rote / 
recall questions. This will have a downstream effect on the nature of school test questions and classroom questions. 

4. Teacher training that impacts the teaching methodology to make it more understanding rather than rote-focused will be a critical 
enabler. 

5. The practice of Boards to give students very high marks - close to 100% in most subjects - has many negative effects. Apart from 
creating pressure (because a single mark matters so much more in a high scoring situation), inflated scores are often taken as 
indicators of thorough learning. Boards should start awarding students a percentile score in every paper. This is neither difficult to do, 
nor is it objectionable in any way. 

6. India should participate in international benchmarking assessments like the TIMSS which test students on how well they can apply the 
learnt competencies/concepts. This will help the country benchmark student performance with the rest of the world and become the 
basis for improvement. Benchmarking studies like the TIMSS can be conducted within India also. 


O 


© Educational Initiatives Pvt. Ltd. 



■ DISCUSSION - ARE STUDENTS TAUGHT THE CONCEPTS THAT WERE TESTED 


The NCERT mathematics textbook of class 2 carefully shows how the length of an object can be 
measured with a scale. However, we see that a lot of students having trouble measuring the length of an 
object correctly even in classes 4 or 6 . 


r\ 
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VHiatlbe class 2 texttoult says. 
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Source: NCERT, class 2 page 178, 'Measures of Length * 


...various questions involving practical measurement. Here is one example: 


How many 
centimetres of string 
is hanging below the 
balloon? 




A, 

about 20 cm 

B. 

about 45 cm 

C 

about 60 cm 

D. 

about 65 cm 


4 



x - Classes tested 
y - percentage of students who 
answered cor reed v 




*h GX 

b m 

*t 13K 
*0 421 


4 9 12 16 £0 £4 33 32 


jf - Ability of students 
y - percentage of students who 
answered correctly 


It looks like a majority of children (49% in class 4, 42% in class 6, 25% in class 8) are learning measurement 
mechanically and choose option D. When measurement does not start from zero, as in this question, they do 
not see a need to subtract the starting point from the final one (in our case, 65 - 20 = 45 cm). 


■ REFLECTIONS 

This study shows surprising results that even in the ‘top’ schools, a significant fraction of middle school students have not mastered primary 
school competencies; and that significant differences exist in the learning levels in various Indian cities. We would like to emphasize that the 
purpose of this study is not to sensationalise these gaps or even to identify the so-called ‘top cities’ or ‘top schools’ - in fact, these aspects 
have been consciously downplayed in the report. The question of interest is how can we discover underlying, non-apparent patterns and use 
that learning to improve the system? That is the purpose of this study. 

We would like to believe that studies like this help us identify problems and thus give us a chance to move onto the cycle of problem 
discovery, cause identification, solution implementation and feedback (which may lead to a deeper level of problem identification) and thus 
create a positive spiral towards improvement. 
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■ ABOUT El 


Educational Initiatives (El), the makers of ASSET and Wipro Applying Thought in Schools jointly conducted a national level 
research study to understand how well children are learning in the top schools of our country. Over 32,000 students from 142 
leading schools of five metros - Mumbai, Kolkata, Chennai, Delhi and Bangalore participated in the tests conducted between 
February and April 2006. The November 27 th issue of India Today carried a cover story highlighting some of the main 
findings of the study. The full report and the detailed questions-wise analysis can be downloaded from the following link 
http://www.ei-india.com/whats-wronq-with-our-teachinq/ 
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TEACHING? 


We are an educational research organization that focuses on learning research through assessments. El has been started by a group of IIM 
Ahmedabad alumni with first-hand experience of setting up and running educational institutions. It has been formed with a mission to work towards 
qualitative improvement in India’s educational system and our vision is “A world where children everywhere are ‘Learning with Understanding’”. 

PROJECTS OF El: 

Andhra Pradesh Randomised Evaluation Study (2004 onwards): Done in partnership with Harvard University, Azim Premji Foundation, World 
Bank and the Government of Andhra Pradesh, this is a longitudinal study across 8-9 years and covers currently 100,000 elementary school kids 
and measures the impact of various inputs (eg., block grants, additional teachers) with outcome-based teacher incentives. 

Assessment of Student Learning in Sarva Shiksha Abhiyan - RGSM, Chhattisgarh (2008 onwards): The test was developed in Hindi and 
administered to approximately 3 lakh students in about 1900 schools in 16 districts in Chhattisgarh states. The tests have already been conducted 
for students of class 3 to 8 for Language and Maths and the report is in final stages. 

Municipal School Benchmarking Study (2004-2007): Supported by ICICI Bank, this study assessed learning in 35,000 municipal school students 
from class 2, 4 and 6 across the 6 biggest towns in each of the states - Gujarat, Andhra Pradesh, Rajasthan, Chhattisgarh and Uttarakhand. 

UNICEF Learning Assessment Study for Quality Education (2005-2006): assessed mathematics and language acquisition among primary 
school children in the UNICEF quality package schools in 13 states of India. The tests were standardised across 9 languages and involved very 
intricate development cycle involving language experts from all over India. 

Teacher Needs Assessment (2008 onwards): is a census study that has been initiated by the Royal Education Council, Government of Bhutan. 
In this project all teachers of Bhutan are assessed for their general ability, competence in subject knowledge and pedagogical practices. 

Student Learning Study (2008-2009): Supported by Google Inc., this study is currently ongoing and assesses student learning in 21 states of 
India. Nearly 190000 students in classes 4, 6 and 8 are tested for learning in Language and Maths in rural and urban govt, schools. 

El’S PRODUCTS AND LEARNING SOLUTIONS: 

ASSET: is an objective-type, multiple-choice test for students of Classes 3 to 12. It is a scientifically designed, skill based assessment developed in 
India for Indian schools. It assesses students’ level of proficiency in the skills and concepts underlying the school syllabus and provides them 
feedback about their strengths and weaknesses. Know more about ASSET at www.ei-india.com 

Mindspark: is a computer based self-learning programme that helps the child improve his/her skills. It allows each student to follow a learning path 
that is based on his need. Mindspark is currently available for Maths for classes 2-10 in English version. Mindspark can be accessed at 
www.mindspark.in 

Rural Mindspark: Hindi version is currently available on demand for some Maths modules. Contact El to know more about other language versions 
and modules. 
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